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Foreword - ■ \ 

In June 1 980, the Illinois State Board of Education initiated a pfart for a Decade 
Study to compare the academic performance of Illinois high school juniors in 1 9,81 
with that from 1 970. tn adfiition-tq collecting data on academic achievement,, the 

o plan called for other types of information about students and their environments. 
The purpose of the study wpre d ) to compare performance on a testof Natural 
Science Social Studies, English, and Mathematics in 1 981 to performance in 1 970, 

* and (2) to identify the characteristics of student,. home, pr School related to the 
results. A. \ ». 

The baseline test information from 1 9.70 was made available to the state agency by 
, the Center for Instructional Research and Curriculum Evaluation (CIRCE) at the 
\ University &f Illinois, Urbana-Champaign.'Cqrpparative information about student \ 
performance in 1 98 1 was obtained throu^festin^ a random sample of.the same 
schools that participated in 1970. * , ' ' o' ... 

r\ , ' V . ^ f A \ ^ V ' 'J 

The implementation of this study requlreothe cooperation and effort of rjibny . y 
people, including principals, teaqhors, cdunseloffc, and students. The support of * " 
Thomas Hastings at dlRCE allowed tho project to get its start. Permission to use the 
test was granted by Educational Testing tpervico (ETS\in Princeton, Now Jersey, 
whore Jack IVTqe was of groat assistance. At tho University of Florida^Robprt 
Foinborg provyifldiaSsiatance in obtaining jnf ormation about tho dovolcftmont of tho 
test and its uso in FlQrida. At'tho Illinois Stato Board of Education, conceptualization, 
'design, and implementation of the study were undertakofoprirparily by Norman Sten- 
zel and Leslie J. Fyans, Jr. In addition, an advisory, committee consisting of Thomas 
Hastings, Robert Linn, and Delwyn Harnish of the University of Illinois, and Roger 
Farr of Indiana University providod fiuidonceUn the p tonninq stag e j)f the project/ 
This study would hpve been impossible without the enorgy of these and, many 
others. ^ 



This Report on student, achievement in schools is 
intended^ to^ provide ^formation to the Jllinois 
State Board of Edpcatioft and school district staff 
,^ in lllinojs? The i;e|£drt summarizes the results of a 
, study of the achievement of Illinois high school 
i juniors -ip 197^ and 1 r 981. $he purpose of the 
•stucf^ weCP ty^ provide ^comparison dfXstudent 
J?il(3erforrnanco-oyer a period df time artd to identify', 
rf edufeatiorial, social, and per^oru^ conditions that 
v relate to performance, irv t^ese r&spects, the study 
\ was-intended to add aftevr^dimension to informa- 
: tion about the current pbndition of education in 
the state. ' V & >" \ 1 

tievefc 



v systematically cdllecrted then. This is a limitation 
of this^study." Although the study cannot pinpoint 
- all of the conditions contributing to differences 
from one set of results to another, it does strongly 
suggest that some c; ^rafc f eristics of student?, 
* their homes, and/schoo.ls sre more likely to in- 
fluence academic performance than others". 
s THfe majo^fipdi^B of this study are: , ' 



iment* Was Nrieasured byua~battpry of 
tos*ts originally designed as a" college entrance 
Anamination by Educational ^esi&ig Sorvfce of - 
Princeton, New Jersey, the Jairttery included sub-' 
teste on English^ (Vlothornqtics, Social Studies, and 
Natural Scienco^n order tQ ehsure^m^c^nparabil- 
ity of infpri^iation, tes^ng in |981 watf conducted 
in a npannor similar to that for 19?0. Students ir^ 
1901 wpre /ested in a sample 6* 1^2 of the 307. 
•publie-schoote .where testing t$6k plac&jr) 1970^ 
wheft ttte tbst w as offered a s a soiyico JjyJthQ\_ 



Renter for Instructional Research a and!.Surriculum 
Evaluation at the University ^ Illinois, Ur&ana- 
Champaign. -The number of student records used , 



in the analysis was 1 1,4-eajn 1972 r ahd 9,64J \jn? 



1981. V- . 

- Additional information to help explain tbst re- 
sults was gathered for the study in a number of 
ways. Information about ; the sphoo$ aqd theit set- 
tings for each \bf the times represented in the 
study wqs taken from thojrocords of the Illinois 
State Board of Eduj^itign^Armther sot of informa- 
tion about students, their families and homo 
environments; and their schools was -obtained 
from qUestionnaijdb included in the 1 931 adminis- 
tration of the battery. 

One of the difficulties of a comparative study 
of thia> sort is th&t important inforrrratibn is not 
often available from the past because it was not 



Academic performance of students' in 
Mpthematics, English, Social Studies-, - 
and .Natural Science as measured, by. 
the,*ETS battery A is significantly Jower 
now than it was 1 1 years ago. 

* - : \ 
The decline in performance occurs for 
students of all ability leyels. This is true' 
oven -for «tho top/ (j% . of students, 
except for performance on one Math- 
ematics subtest. / N 

The average level of achievement in 
most schools Was essentially un- 
\ changed from 1 970 to 1 98 1 , relative 
to the rest of tho schools in the* 
sanipty; significant improvement oc- 
curred in 22 schools while 18 declined 
significantly on one or more subtests. 



School characteristics such ps 
enrollment, dropout rate, student- 
to-teacher ratio and per-pupil expend!- 
* turo did not account for a significant 
\ proportion of the differences in perfor- 
mance between 1 970 and 1 98 1 . * 

5. Between 40% and S0% of the dif- 
„ ferences in performahco on the sub** 
tests in 1981 can be explained by a 
combination of thS effects of schools, 

» families, and student motivation. Most 
impotent of the* specific conditions 



us>ed as part of these general catego-. 
*. ries.' are: i the frequency of .talking* to 
"parents about ^school work, the eduna- 
." c tionak level of 'parents, student self- 
7 appraisal of v . mathematics 
*• • performance, and number of math- 

* ematics courses laken. 

\ - • A- ••• <•*•'. . ; • 

Ttie remainder- of the report* focuses fon' the 
issues best reviewed with the information, gath- 
ered and analysed in this study. The appendices 
describe tjfte test?* the study and ~the analyses 
conducted, and the prospects foncomparatlve 
studies at the local level. 



Student performance 

Student performance on the Decade Study battery., 
v^as significantly lower on all of the subtests in 
198ras compared to ^1 970. Declines are most 
pronounced for both of the English subtests'and 
least pronounced for one of the * Mathematics 
subtests.' (See Exhibit 1. A complete ta^le'bf raw 
score results is in Appendix A.) » * 

For 'each of the subtests, the decline from 
1 970 to 1 981 for tfcie average student in terms of 



the percentage of itemd answere'd cQrrectly is 
from 32.5% to 31.7% on the Mathematics I 
subtest; from 38.3% to 34.2% on the Mathematics 
II subtest; from 49'.4% to 43.4% on the English I 
subtost: from 55.3% to 47.0%^on tha English- II 
subtest; from 44.7% to 39.0%*on the Sooial Stud- 
ies subtest; arid from 41 .7% to 38.7% on the Natu- 
rat Science subtest. " " ■ 



• ■ Ability and performance 

. % J j «j 

i The declines in performance generally^ exist" for 
gifted students (the upper 5% of the 11,466 stu- 
..^ents.in'1970 and the 9,643 students in 1981) as 
welJ as for st^dents^ at other levels of ability. Onfy 
0 on the first Mathematics subtest did the gifted 
perform at ajevel similar to the top 5% from 1 1 
years earlier , \ 

■ ; The Jdwest percentage of items answered^corr 
tectly-tfy the upper 5% of students in 1970 and 

* . 1 981 was 65.0% and 64.4% on the Mathematics I 
, subjqst; 70.0% and 65.0% on the^Mathematics II 

subtest; 77.4% and 68.6% on the. .English \ 
subtest; 81.3% and 70.7% on Ihe English II 



subtest; 77.3% and 69.7% on the Social Studies. 



subtest; and 71.8% and 67.8% on the Natural Sci 
ence subtest- ^ 



School characteristics iind > - 
performance ~„ 

'School averages were formed oh the b&sis of stu^ 
dent scores in each' school. The results show a 
small number of schools improving or decliryng 

.significantly. Twenty-two of th'e 122' schools 
.tested improved in performance, whereas 18 de- 
clined significantly on one or more of the subtests. 
(See Exhibit 2.) * 

Most frequently the changes represent perfor- ' 
mance on only one or two of thie subtests. 
HpweVer, in three schools f performance was signi-, 

* f icaptly higher for three or more subtests; one 
school improved in five out of the Six subtests. De- 
clines in three ormore subtests occ6rred in three 

^schools. One of those schools declined to a sig- 
nificant extent on five out of the six subtests. 

The characteristics of schools in 1970 and 
1981 available for this study included secondary 
enrollment, dropout rate, student-to-teach^r/atio, 
, and per-pupfl expenditure! These features. did not 
account for a sizable portion of the difference in 
'scores from one year to ther^xt in Mathematics, 
Social Studies, or Natural Sciehce, but (decreases 
in - English performance-, from 1970 to 1981 
tended to occur where there were increases in pef- 
pupil expenditures. - (~\ 

Although the, characteristics studied do not 
a&ount for a-large proportion of the performance 
differences, this does not suggest that school 
characteristics are~ncTf part^Ohe explanation foF 
declining scores. State office information collec- 
tion in- the past did not allow course offerings, 
> course" content, course - enrollment, teaching 
methods, or other aspects of the school setting to 

* be included in the analysis. 



Student characteristics and ' 
performance 

In addition to tfjd intorrtiaticn about the features 
of schools that was available for the analysis of 
both the 1970 and the 1981 results, t information 
about family characteristics and student motiva- 
tion was gathered as part of the 1981 administra- 
tion of the Decade Study battery. Together these 
variables help to ^account for between 40% and 
50%of the differences in scores in J 981. *~ 

Family background information included the. 
level of parental education, family size, and stu- 
dent communication with parents about, 
schoolwork. The information about student moti- . 
vation included questions about, self -assessment 
of performance*, ease in-takinjg tests, and *tn^ stu- 
dent's perceived, value of achievement.' 

There -are differences in the relationship of 
these three types > of informatiorrto student perfor- 
mance on eactvof the subtests. (See Exhibit 3-j 9 

The analysis clearly re veal qd 'that sfchool con- 
text and family background had their strongest ef- 
fects on performance on Er^tish I, English II; and 
Social Studies. Student motivation was most in- 
fluential for. Mathematics »l and MathGhiatics II. In 
mathematics, student motivation has greater 
impact than the effect of either schodl context or 



family background. Performance in Natural Sci- 
ence in 1981 was affected in roughly equal jneo- 
/sure by farrtily background an^ student 
7"motlvatTon; less~Tnfluwce — was"~dtre7to"school 
^ context. Detailsof these results are in Appendix A. 
The nature of the items constituting the three 
types of features may influence these results. The 
strong relationship of mptivation to mathematics 
may reflect a nupiber of motivation questions that 



^ . EXHIBIT 2 , J'- ^< " ; . : -. . ; : :% « , h"- - ; < \;, 

• Nurftbor of aclioolo whoro port ormanoo ohongoddighlf toantly, 1 870-1 00/1 
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influences on subtest performance in 1 981 
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specifically* mention mathematics. Family in- 
fluences appear to relate to verbally orientqd areas 
.such as English and Social Studies. Students who 
talk at home witfoswell-educated.parents may have 
an advantage in learning about other cultures and 
torrect language use. " **' ' f 



Effects of environment and 
motivation 



The school, family, and* motivation features 
grouped together for analysis of performance 
were also examined as individual conditions that y 
could relate to student achievement. Specific char- / 

; : acteristics of these three elements are related to / 

] : student achievement. Some relate to higher I 
performance, while others relate to lower. perfo£-| t 
mance on thiTDec^e~Study^ 

' The, ? aspects of school context— secondary 
enroNment, student-to-teacher ratio, dropout rate, 
and per-pupll expenditure— all have particular rela-^ 
tioftships to student performance. Both secondary. 

? Nschool. enrollment and dropout rate are consistent-'** 
Jy related ( to student achievement^ In this sample 
of schQols, students from schoqls'Svith more than '. 

! the sample ^average .of 435 students performed 
significantly higher than students from smaller 
schools. In respect to dropoi|^?ate, schools v vvith 
less than the 4% sample average have significantly 
* higher achievements For the final two .aspects of 

< context, student-ttdteacher ratio and per-pupil 
■ expenditure, the influence is .not consistent. For 
exarripje. in terms of the.student-to-teacher ratio;, 

! % higher achievement in Social Studies and, English, 
is associated with schools with a ratio smaller 

| than the 14.8 to J 'sample average, while the • 




reverse is found,for the Mathematics I subtest. 

>p * Family conditions also are related to student 
achievement. 'Students who talk with their parents . 
abdut scboolwork perform better than their peers, 
in English, Social Studies end ' Natural Science. 
The amount of communication is clearly one of . 

fthe most significant positive cqntjibutioYis pf the * 
famify to a student's education. Thp education of 

. the parents, both father and mother, is positively — 
related to high achievernent'in Mathematics and 

-.-'Social Studies* • * t < {v^Lj 
1 '.Differences' in agli ane^se? are factors in?\3st 

/results. Students 17 or older had h1gher)achieve- 
m§rit than younger students pn the second 
the English subtest. Males scored' higher ir 
Science and Social jStUdies than femafes; whUe 

" females scored higher on the English II subtest. 
In terms pf4h N e motivational information used 

— in-the-studyrttoe most-positive predictor-pf^tudent-L 
achievem'ent was <self-appr^sal *of ,ih^ perfor- 
mance students expected on the Mathematics 
subtests. Students performed clpsely to their own 
estimates of ability. A .second important piece of 
motivational information is the number of math- 
ematics courses *taken. This- information indicates + 

k whai has been called intrinsic interest, thirst for 
knowledge, or continuing motivation. The. more 
mathematics courses, the students fake, the 
higher their achievement in Both Mathematics and 

. Natural Science.. ; * 



Discussion * "* M 

•Public interest in the level of student performance 
is periodically fueled by reports of declining 
achievement test scores. Often thoss reports are 
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based upon tests th^at reflect only a small propor- 
- tion of students in Illinois.. The Decade Study has* 
avoided that limitation by using a more-representa- 
tive population of Illinois students. Consequently, 
* the Decade Study has allowed us to evaluate the ? 
outcomes- of schooling during the junior year of 
high-school by comparing students* of a decade, 
pgo vyith contemporary students. 

The results of the Decade Study indicate, that 
the performance 'of high school juniors* wassigni- 
t ficantly Tower in 1981 than it was in 1fl70. Indeed f> 
the.decline was general in all subjecjLareas tested 
and for all ability levels of /students,. e*<!tept the top 
5% of the students taking one. of the Math^ma'tibs 
" subtests. The decline in student achievement was 
■". ndt>elated to changes from 1970 to 1981 in the* 
enrollment, . per - pupil expenditure, student- 
. to-teacher rdfio, ondropouforate jn schools. The re- 
sults aggregated for individual Schools showed 
Ttrot the majority of schools did not qhaftge signifi- 
* cantly in average perforitiance' from* one test ad- 
ministration* to' &te next, ^lowever, 22 schools did 
perform Significantly better and 1 8i perfbrmed'sig^ 
; nificantly worse than in 1 970. "* . 

A profile af characteristics' related tp sfudent 
achievement in 1 981 can. be given for the features 
of scbobj context family characteristics, and stu- 
tt dent rhotivational factors collected with the. most 
..* recent administration .of the test battery- Features * 
•; ^"retlated to school context wefQ moststrdngly relat- 
" ed to peWorm^nce on the*English and Social Stub- 
•v — ies-subtests— The schobl~varibbles-that?rnost criti- - 
'* cally- affected achievefnent were enrollment .and 
..." / dropout- rfite. Better performance in English and 
Social Studies was found in 'schools with 'gnrolU-' 
rnents larger than 435 students and Jower than 
average dropout rates. Neither student^to-teacher 
.Va'tio' nor per-pupil expenditure was' consistently^ 
related to outcomes. Three aspects of family life 
were "also strongly related to achjpaErnent, m 
1981; father's veducatipn, mother's education, 
and talking^ to parents aj&out school Overall, the 
greatest influence of family on achievement was, 
like school condition upon the.students' verbal- 
.' skills ort the English ancirSocial Studies subtests. 

Iii terms of motivation, three features were .strong-,, 
v ly related to the 198J student performance: the - 
students' estimate of success on the Mathematics* 
subtests, th§. number/ of mathematics /courses 
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.-^tjaken «by students,, and students', belief in their 
.own abilities and efforts, to pdVform well in school. 

; Some, of thesfe* features were expected to be 
^important (becjkusd of the Work already done by re- 
► searchers in education? Other features were con- 

• firmed because of the type of large-scale data col- 
lection undertaken for the firsf time in lllinots. The 
Decade Study shows that as researchers examine 

* the issike of how td- identify successful schools or 
v haw to characterize successful^ students, their 
r tf/ork should take into account school, famijy, and 

jyotiyatjon. Pjersons seeking to improve the educa- 
„ tional process will also have to consider, a similar 
scope of conditions. * . 

Plans for school improvement often focus on 

o curricular offerings or course requirements.. An in- 
tensive c^e. study of the curriculum and other 
variables in the 1 schoojs in which, test performance 
significantly increased *&rdecreased could provide 
/ additional , explanatory information, about the 
results. The Decade Stb'dy" does not discount the 
impact of curriculum, but does suggest (hat other 
conditions' arg vital for improved ,test results. 

a Family conditions represented by level x>f parental 
education {reflecting aspirations,^ wealth, . and 

-ability) J arel undoubtedly beyond the control of 

^^ither educators or legislators. On the other hand, 
encouraging^ parents Jo discuss schoolwork with 
their children, may be ah important contribution* to 
achievement./ TJiis would support, in pqrt, the ini- 
tiative of;the National Committee ; f or Citi2ens in 

- Education in 1982 that ' suggested-that~parehtal 
. interest In schooling wotjd positively facilitate 

performance. The rriotivS^on^f. a student relates 
• directly to whether •& student will deliver extra. 
~ *efjprt to complete a task well or to obtain more 
' knowledge in a subject matter area/ The Decade 
Study also dramatically highlights the role of 
motivation. Although student motivation is often 
discussed by educators, it is less oft^n a part of 
school improvement efforts. Tb» best* intentions 
of parties interested in improving student perfor- 

- rnance'on outcome measures will go awry witho 
significant consideration of student'motiv^tion. 

' The-Decade Sjudy sheds li&ht^prKSome unique 
facets of Illinois education^Jt^tju^tful considera- 
tion of this inforrnajion^can contribute to the im- 
provement of education in Illinois, \ 
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APPENDIX A^frME DECADE STUDY TEST 



\ This appendix is provided {for per- 
sorts interested in the nalijre-of 'the 
test used in the study. flomtf may 
contend that the content and nature 
of the test influenced.the outcomes. 
FtfKQxarnple, test questions' may* 
have been more difficult at one tirne^ 

^ than at another because of curricula 1 ^ 
changes or perhaps because of dif- 
ferences in th^ information disse- 
minated through the media. Al- 
though this study examined the cur- 
ricular information available in the, 
state office, characterization of the 
curriculum at the times of the test 
administration or during the 'educa- 
tional, careers of students was not 
possible The following information, 
then, is* provided as a starting point 

. for those who wish to pQrSuo tho 
is^uo. 



Education decided to implement a 
study of student achievement to 
Compare past p^rforrnance to con- 
temporary performance, the test 
used by CIRCE was determined to 
provide the broadest baseline infor- 
mation readily available for the 
project. (See Exhibit A-1.) Permis- 
sion to ? use' the test was obtained 
from CIRCE, ETS, and the University 
of Fiqrjda. ^ 



ORIQIN8 

during tho. late 1 96Qs, tfio Universi 
ty of Florida contracted tho Eiductf 
^ tioniol "Tdsti n^So^ico~ (ETS) " of 
Princeton, Now Jersey, to develop a 
t college entrance examination to bo 
r 'odministoredtetP higl\ichool soniors 
in Florida. On the'basis. of specif ica- 
^tions from Florida, ETS developed a 
• battery of instrument^ that were > 
pijoMestod in 1967. Tho tost was 
charactered at that time as difficult 
by the ETS dovoloprflent staf fr.v 

A few year* later, in 1969. the 
. Center for Instructional Research 
jind Curriculum Evaluation (CIRCE) 
% at the University of Illinois .at 
Urbana-Champaign bogan to looKy 
for a now tost to use in its testing 
service 'for lllinpis high schools. A_ 
^ major portion of tho Florida iost was < 
acquired to bp used for high school 
juniors in Illinois. Tho tost was first , 
administered in Illinois .in 1970 as 
port of the CIRCE service. > . 
When the Illinois State Board*of 




TESTf^NTENT - . 

Although the names oHhtyoubtosts 
In thofbottory oro isimitaK to school 
^ubjects, the general lobols;qf Natu- 
ral # Sc|onpo. Social. Studio*,* end En- 
glish all'servo to poyor moro.apocific 
academic tppics.- for Qxomplo, thii 
Natural- Scionco subtost horns 
included pttysids, chemistry, and 
biology; tho, Social Studios subtest 
included world* history, gfcvornmont, 
and Onitod States history. ^ 

The; Social, Studios subtost con- 
tainod oiglit^ item's about 
government, five items about world 
history, ;throo items on United 
'States jhistory, throe items about 
economics, four items about 



sociology, three items about 
geography, and two items about 
general culture: The format and con- 
tent of these items, are quite varied. 
Sixteen of tl}6 items relate to infor- 
mation presented in the te% Two 
relate to the interpretation of a 

- cartoon, four to information in a 
graph, four to a schematic represen- 
tation of a 1 "national assembly," and 
four to a map of a portion of colonial 
Africa: Jn spite of tho illustrations 
and readings, only four items wore 
directly answerable with the infor- 
mation provided, whereas nine wore 

. onsworoblo through inference. Tho 
17 knowledge-based items in tho 

. Social Studios subtost required stu- 
dents: to Know abbu| tho terms 
"loft" and "right" whon applied to' 
European political part? labels after 
World War I, tho political moaning 
of "radical," general biographical 
knowledge of Muhammad and foa- 

^ turps (if Islam, tho Tonnospoo Valld> 
Authority, and tho political gooarb- ^ 
phy of colonial East Africa, j v 

Two English subtests (66 items) 
4 dealt with what are generally called 
editing skills: Although doth of the 
subtests included some similar 
confont, performance fl on bach /of 
the sections differed slightly, the 
ETS description indicated that six 
itoijis troa'tou wording and 
expression, eight dealt with idioms;. 
■J eight wore about parallelism; twelve 
^ doajt with modification^ . six: ^oalt 
with logic and cohoronco^ sixtooo 
doalt with subject-verb agreement; 
and nine applied to pronoun use. 4 

% Tho two English i/sago subtests 
measure similar knowlodga. about 
correct grammar, /but within, dif- 
ferent formats. On tho English Part I 
/subtest, students wore required to_ 
roajj through a 7 ,; sontonco ib find 
errors in any of the underline* parts. . 
-Por oxomplo: / . . 



\ 



He work s every ^lay so that he 
A -J B 

V ,n ■ 

would befcome . 1 
financially independent 

■: • • ' 

in his old-age. N6 erroc 



' (The correct answer is "A.j) / '' 

* ■ ' •■ .1 

Most of the students who failed ' 
tp answer these types of questions 
^correctly responded "E" or "No 
*rror." None of the other response 
alternatives were chosen to any sub- 
stantial degree. This is^a typical pat- 
tern for many. of thtntems in Part I. 

'On the English PartMf subtests, 
students were requited to'demons- 
tratq ^ssentiaHy the safne kqowl- 
edge'about correct grammar as on 
ParUI, but^were required to correct 
ttfll underlined section of each 

* semence. An example ' of this 

* second format using the same ccfn-. 
tent as' ilv 'the previous example 
follows; 

"•".•:'» 

He worlts every day so that he 
would become financially /. !» 
independent in his old age/.;/ 

i (A) Me .works ©very day^sfc that he 
k -would become r ■ . ' ' 



(B) 



(C) 



He worked every, day so that he 
would become v : v. ' ^ 



Me^ worked eve^g day v in order 
that he would become ~ 



(D) .Working .every v 
* , _ w oul Id i?ecpjria 



(E) He had 




worked- every 



^ , becoming * 
' (The correct answer is 

" o • : i\ . ~ . i ■ 1 • h ' 
t * The Mathematics ^subtests . 

iribludod 60^ items representing 
, arithmetic (7 litems)/ algebra^1&/ 
° items), geometry (1 .2} items), dofink* 

tions (8 itpms), set theory (4 items), 



graphs (6 items), and other topics 
(5. items). The presentation of the 
mathem'atjes questions included the 
workbook and story problem styles 
with multiple-choice answers from;, 
which to'chGOse. Seven of the 60 
' items were story problems. The ter- 
minology of mathematics plays an 
important role in understanding 31 
of the questions. Such 
mathematics-oriented vocabulary 
and the language of the items 
included "congruent sectors," 
"isosceles triangle," "scientific 
notation," "intersecting planes," 
/'fourth -degree polysomia I, 
''multiplicative inverse," 
"associative /law of addition," 
"irrational number," and "base 1 6." . 

The Natural Science subtest con- 
tained 23 '-items covering physics 
(10 item's),*- biology (7 items), and 
chemistry, (6 items). The natural sci- 
ence items included, 1 1 items, that^ 
'defended on reading ability. Five ot 
the reading iternsAould be answered > 
x directly -on the basis of fhf ormation J 
in thecpassage, wfiile 6 others coulp n 
be : answered through,* .inference j 
from [n'tprniiition given iiOthe^texft.j 
The remaining $12 items required j 
. the student to be fdmiligr with i 
V specific knowledge to answer the 
^qupstion. Reading topics included 
v' types of parasitoa?, . the c Efansen 
' burrier, and (Wax ' (Jlanck's theories 
about guanta/.*Spociffp knowledge, 
, items included thtf definition of the 0 ' 
7 "vector sum" of forces, the nature 
. of . transmission of yellow, foyer, a 
product of ipcomfjfete opmbustion, 
an eaampje of electromagnetic 
radiation? tVfo general 
the operation, of an electron 
mictoscopejJand the definition of dn 
empirical law. $, : y 
t J Tho\1981 test had a ton-item; 
. s u btbst labeled jaQ "Gonoral Skilla/', 
which did not appear in the 1 &JQ- V 
tost. Ttiat lection included items' 
frpnrtherlllinois- Inventory ;,;pf Educa- 
tional) Progress (HEP); the state as^ 
- .sesspient, program. ; i Xhis->^fiection' 
a contained .mathematics :and Trading 
litems of moderate difficulty: 

TEST CHARACTERISTICS . 

«• ; ) ' . - •-; : 

A number of, technical, characteris- 



tics of the battery used^ ip .the. 
Decade Study were examined. 
" Factor analysis was us8d to examine 
the structure of the te§* oh the basis 
^ of 1981 %tjjdcn| /y performance. *The 
1981 data were .also /examined 
throughya three-parameter ' logistic 
program, to identify a • number of - 
item characteristic^ /in addition to 
subtest dif f iculty levels! Subtest reli- 
abilities were examined ysing S p|jt- 
tialf techniques.^ V 

Factors J . 

■ % ■ . ■ •■ 

Results of the alpha-factoring, ap- 
proach indicated that one factor ac- $ 
counted for over 50% of the total f 
test variance. This factor included a 
number of items from; the English^ • 
5 usage tests. The first four factors in 
ord^r of strength were English 
usage,. mathematical reasoning and , 
?/ definitions, social studies and^ 'sci- 
ence reading, and English grammarf 
. ; - • ' •/- 

Difficulty & 

The beta .values iffjbrn . the logistic.'; 
*. analysis shovv both that the Docodb 
Study battery was a difficult test as 
a whole and that aoms subtests aro 
more^ifficuit ?han others/lri thCLf ol- 
lowing list bf>aiuo8, o t3.0pj5 diffi- 
cult arid -3.00 is eaay:>TbS QqnDral 
Skills subtest "ropTo'sontftd the iotfbf \. 
,, 10 anchor jtomsi from thti IIEj* that 1 
<ywere included ,to detDrmino Jt\Q 
.■ goneralitabilityjof sample rosjulta to 
> statewide dimensions. 

jtromagnotic ^mf^^'^rri} r>^i^%fe 
principle of ; ip*M|felTW 
n electron' i ^iitiiil* lit • 
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Another perspective about the 
difficulty of the battery is obtained 
by examining the, formulas which 
show relationship , between 1970 
and 1 981 scores. The ektent of the 
decrements between 1970 * and 
1981 is expressed in the following 
weighting formulas: , 



/Social Studies, 1970 
Natural Science, 1 970 

£ riglish 1, 1 970" 

English II, 197b 
Mathematics l f 1 970 
Mathematics II, 1970 



1.13 (1981) 
V.04(1981) 
1.04(1981) 
1.02(1981) 
1.02(1981) 
1.06(1981) 



+ 
+ 
+ 
+ 

.+ 
+ 



v 



... Using, the formula for Social 
" Studies, for example, shows, that a 

score of 17 in 1,970 would have 

been 15 in 1981. 

Reliabilities 



.22 
.34 
-1.41 
2.17 
.13 
.50 



, The internal consistency of the sub- 

^ tests on the 1^81 Administration 
, was slightly tower than is usually ob- 
tained in standardized testing. Relia- 
bilities ranged from d\ to .81 with 

. the Ruder-Richardson formulas 20 

. ' and 21. (See Exhibit A-3.) 

Reliabilities ^vere undoubtedly 
low on the general skills materials 

. ' because t^e general skilly instru- 
ment contained only ten items, and -y 
reliabilities generally' improve with 
length. For the other subtests, the 
number' of students operating at a 
near-chance level could be a con- 
tributing factor, o ^ _ 



Performance 



-\ 



, Results of performance on the 
\ .Decade Study battery were present* 
; \ ed as part of ah Illinois State Board ' 
Aof Education report entitled Student 
"• Achievement in Illinois: An Analysis 
ok Student Progress.. November 
1982. The^ Exhibits A-4 through 
, A-S^aresent alternative formats of 
the results, 



^EXHIBIT A-3 ^ ;t: = , ; r ^ ±\ f' *. V \ 




STUDENT ACHIEVEMENT, 1 970 AMD 1981 



EXHIBIT A-4 



Raw score results for all students 



Subtest 


Number of ' 
;testjtems 


1 970 mean 


1 970. 
standard 
deviations 


1981 mean . 


1981° 
standard a 
deviations 


Difference 
between means 


Mathematics 1 
Mathematics If 
English! 
English II 
Social Studies 
Natural Science 


, / 
36 
^24 
35 
30 
30 t 
• 23 


/ 11.7 
/ 9.2 
17.5 
16.6 
13.4 
9.6 


5.9 
• 4.2 
5.7 
S 2 
5.5 
3.8 


11*4 
8.2 
15.2 
1 4.1 

11.7 ; 

8.9 


. 5.9 
3.9 
5.5 
5.1 
4.9 

3.7 , 


- .3 
-1.0 « 

* -2.1 
-2.5 
-1.7 

- .7 


EXHIBIT A-5 

Raw scores results for groups of students 






/ 




y 


Number , 
of Items * | 


Upper 5% 


/ 

Upper 25% 


Upper 50% 


Upper 75% 






X ■ | 










Mathematics'!, 1970 " 
1981 


*> 


36, 

1 


23.4 
23.2 


• 15^2 
14.5 ~ 


10,5 
10.0 


7.3 - 

7.1 


Mathematics II, 1°970 
* - 1981 


> 


24 


16.8 
15.6 


12.0 
' 10.5 > 


8.7 
7.6 

i 


6.1 

5.3 


English 1/1 970 * 
1981 


0 


35 


27.1 
24.0 


21.3 
° 1 7.9 


17.1 
14.8 


13.0 
1 1.2 


English 11,1970 
1981 


r 


30 ' ■ 


24.4 

919 


20.4 
143 

* 


17.0 
14.3 


1 3.1 

10.2 s 


Social Studies, 1*970 
1981 




30 ) 


23.2 
20.9 


17:3 

14.7 


12.9 
11.0 


.18.1; ' 


Natural Science, 1970 
. 1981 


« 


23 


16.5 . .r 

1 5.6 


12.2 
11.4 
— * — ■ 


9.3 
8,6 


|6.8 « 

f k.2 

i - 

L t • 


/ * * 

EXHIBIT A-6 ^ ' ' ' * • w * : 
Row score rooults : school averages, based on 1 22 schools 


* 




* 


Subtest , , 


Number of . 1970 
testitems^ mean 


1970 
standard 
deviations 


1981 
mean 


'198jl 
standard 
deviations 


Difference 
between 
means 

/ ■ • 


-J 

Mathematics I 1 
Mathematics II / 
English 1 / 
English II 
Social Studies 
Natural Science . 


36 
24 
35 
30 
- 30 
23 


11.8' 
9.0 
17.4' 
16.6 
1 3.3 
9.9 


"1.6 
1.0 
1.3 
1.2 
1.4 
1.1 


"\ 9 

\ 

\ 11.1 
\ 8.0 

15.3 
13.0 
. 1-1.7 
7.9 


1.5 
.9 
1.5 

i.a 
1.2 

.8 


- .7 ■■ ~ 

•-'1.0 

-2.1 

-3.6 

-1.6 

-2.0 

0 
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APPENDIX B- DESIGN AND ANALYSIS 



J Additional information about the 
design and analyses of the<s£)ecade 
Study is given here. Although it is 
not "anticipated that the stydy will 
be replicated, some of the specifica- 
tions below support the claim that 
the results generally represent the 
condition of education in Illinois. 



Purpose 

* 

The; purposes of the Decade Study 
were to determine (1) how well stu- 
dents performed in 1981 compared 
to 1970 on the same test battery, 
and (2) which o| the available varia- 
bles characterizing student home, 
and school were related to those 
results. - 



GhnrflolGflstloacftfiohoolo 



Original Administration of the , 
Tost ' - 

From the late 1 940s until 1 976, the 
Center for Instructional Research 
jand Curriculum Evaluation at the 
University of Illinois, Urbana 7 
Champaign, provided a testing ser- 
vice for high schools in the state. 
Participation in the program was 
self-selected by* school 
administrators. 

In 1970, -307 of the 586 public 
high schools in Illinois used the test 
for approximately 34,000 juniors in 
the fall of that year. Testing was im- 
plemented locally in two or three 
sessions over one or two days. Test- 
ing generally occurred in October 
and November. Schools received 




student stores and school level per- 
centile V° rms as* the report of 
results. 



Comparative Sample 

; The Illinois State Board of Education 
obtai ned the list of schools that 
participated in the 1970 tasting 
program. A minimum sample of 
120 schools was heeded. One 
hundred thirty schools were ran- 
5 domly selected from the original list. 
7- 1 — Rve-were-eliminated _ due-to^their~ 
consolidation with other schools or- 
their choice not to participate rTfiree 
. schools did not^ administer the test 
as hacLbeen expected. The reasons 
given for nonparticipation by the 



\ 



1970 



1801 



Porcontago of urban 
population baaed 



Rengo 
Avoraeo 



184 - • - 



majority of the dropouts was that 
the three : hour administration time 
would not fit into their November 
school schedule. The final sample 
consisted of 1 22 schools. ~ - 
The schools were spread 
geographically and were diverse in 
size. The data Hn Exhibit B-1 char- 
acterize the schools at the times of 
the comparison. 



Analysis 

Both descriptive and inferential 
statistics were used to analyze 'the 
data used in the Decade Study. The 
analyses included significance tests 
for differences, discriminant 
analysis, and* standard multiple 
regression. 
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APPENDIX C — THE PROSPECT OF COMPARATIVE STUDIES 
AT LOCAL EDUCATIONAL AGENCIES 



Although comparisons of student v 
achievement such as the one report- * 
ed in this document have been con- 
ducted from time to time on«a large 
scale, it may well be that the most 
appropriate level of implementation 
is at the school-district level. State- 
wide reports include large numbers 
of students and can claim the 
weight and breadth of the popula- 
tion Involved as an advantage, but 
many challenges to such a study 
can be resolved only on a district- 
by-district bdsis. These challenges 
include: % 

• Does the test match past and 
present curricula? 

V 

• Was the match with the curricu- - 
lum better at one time than the 

•other? 

■ «■ 

What proportions of the popula- 
tion enroll in courses. where the 
tested content would be 
taught? . s ■ 

°. 

• Has the population attending 
the school changed? 

v.* ^ 

With .periodic comparisonsat the 
local level, differences, changes, 
and characteristics' can be tracked 
much more precisely than at the 
state level. 

Such comparisons • not only 
should include a review of the test 
results, which many districts are 
likely to do; but, should attend to the" 
conditions of education that are 
likely to influence performance. 
Sofne of the typos of collateral data 
collected for* this statewide effort 
would be useful at the local 
level — motivation, curricular 
exposure, or family > background. 
Districts could add pedagogical 
practices, , district demographics^ * 
course content, and cocurricular op- 



portunities such as science clubs or 
mathematics competition. 

The following list can serve as a 
guide for school districts interested 
in conducting their own comparative 
study. 



COMPARATIVE STUDY GUIDE 

\ 

1. Comparisons should be made 
for all students at a grirde level. 

— A sample including all stu- 
dents capable of taking the 

\ test is best. \ J 

/ — Individual students should 
not be systematically 
excluded. ■ * 
» 

Individual students should 
not be randomly excluded. 

— Illness or absences are ac- 
ceptable omissions. 

« 

* 

2. Tests used in a comparison 
should be the same or an alter- 
nate version of the same test. 

— .If not the same, tests must 
> .be of comparable difficulty 

and statistically equatable. 

^ ■ If not the same, tests must x 
be designed to be adminis- 
tered within a three-week 
range of time at the same 
time of the year. 

3. Tests used in a comparison 
should be administered at the 

. same time of the year within a 
three-week period. 

4. The same metric must be used 
in the comparison; grade- 
equivalent scores must not be 
used. 



5. Scores must be compared 
within curriculum areas. 

6. Collateral characteristics of 
setting* classroom practices, 
students, or home should be 
collected so that the informa- 
tion represents'the same defini- 
tion at 'both times the ^tudy is 
conduced. 

- ° -■' ■ .' v ; . • it 
, — cStudent-lbvel information 
can be collected in different 
ways^from. school, 
records or from tfW 
* student — but should be 
compared if the collection 
method is the same or the 
result is verifiably the same. 



7. 



8. 



— To be of use, classrocin- 
level, grade-level, or school- 
level information must 
become a part of the 
record on each student ifv 
the study. v 

— Family information should 
be collected in the most 
reliable manner. Younger 
students may not Be able to 
report some types of infor- 
mation»about their families. 

— Information concerning per- 
sonal, attitudes', student 

* motivation, and .values 
should be considered confi- 
dential and secure. 

Statistical comparisons should 
be rridtle by standardizing each 
student score against the mean 
of each particular year and then 
. comparing standardized scores 
from year to yeatV* 

Comparisons of standardized 4 
scores can be based ori^he rule 
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that a .1-1/2 standard unit dif- 
-tefence is significant. 

9. ^Achievement and collateral in- 
% formation can be compared 
using standard multiple regres- 
sion techniques. 
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APPENDIX D- HISTORICAL CONTEXT 



Any comparison of educational con- 
ditions at two different times must f 
take differenced inthe context of 
the enterprise into account. The dif- 
ferences in schools over 11 years 
could very * well contribute to dif- 
ferences in 1 the performance out- 

t comes on tests. At a, minimum, a 
history of the educational context in < 
1970 compared to 1981 could be. 
reviewed for potential influences on 
schools. Even that-rnhy not be com- 
plete enough, however The educa- 
tional y experience of the class of 
1 982 was just beginning when the H 
class of 1971 took the CIRCE, bat- 
tery in the fall of 1970. Similarly, 
the, educational experience of the 
class of 1971 bo^an at the end of 
the 1960s., In this serffie, the 
Opcode Study compares two educa- 
tional generations. 
/ This period included the bog in - 
ping of many fnajor programs of 
/fodoral support for education., The 

/Notional Defense Education Act 
was promulgated as one response 
to the 'Russian launch of Sputnik. 
Programs for academically superior 
students appeared to be necessary, 
to moot the Soviet load. By the late 
1 960s other concerns were 



^targeted. Compensatory 
education— education designed to 
overcome the deficits of the 
"educationally disadvantaged"— 
was initiated in the early 1 970s as 
Title I of the Elementary and Secon- 
dary Education Act. "Other special 
concerns brougnt before educators 
included environmental education, 
drug education, dropout prevention, 
education for the handicapped, edu- 
cation for multi-language studentb, 
and desegregffHon. With the election 
of President Reagan, fodoral policy 
bocarne,tp limit fodoral ihjtiativesln 
education. \ 

In Illinois the era includod\chool 
consolidation; legislation* parallel to 
fodoral legislation in compensatory, 
bilingual, and handicapped 
education; legislation loading the 
notionJn giftod educatiori; and legis- 
iation promoting reform, in addition, 

vtho Illinois Board of Education initiat- 
ed efforts to promote long-range 
school planning and review, school 
review by state office staff, and ap- 
proval of toaqhor oducation 

, programs. a 
« At the school-district level, there 
.wore pressures by fodoral and state 
programs and critics. Schools initiat- 



ed consolidation efforts, established 
cooperatives for education of the 
handicapped, implemented federal 
and state legislation, engaged in 
writing 'measurable objectives for 
' school plans/ and .often attempted 
to pass referenda in the facie of 
public opposition. Local -educators' 
facpd "contrasting, circumstances 
during these 22 years" Conditions 
changed from concerns about build- 
ing enough classrooms -for baby 
boom children to closing c buildings 
in. the era of dqctinihg enrollments; 
from teacher shortages \o o surfeit 
of toacher&v seeking jobs,* and from 
emphasis on advanced programs to 
an emphasis on'baoics. 

The responses of schools to 
these conditions form part of the 
background rolotecHQ studont per- 
formance on teats. Examination of 
tho impact of changing conditions 
has not boon undqjiakon In this 
study, but it sooms iikoly .that of 
least a portion of tho difforoncos in 
studont performance is attributable 
to tho difforoncos bqtwoon a Sput- 
nik gonoration comparod to tho fol- 
lowing generation oducatod whon 
other intorosts^ond concorhs woro 
pramiriont .* 
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